A clustering-based competitive particle swarm optimization with grid ranking for multi-objective optimization problems

The goal of the multi-objective optimization algorithm is to quickly and accurately find a set of trade-off solutions. This paper develops a clustering-based competitive multi-objective particle swarm optimizer using the enhanced grid for solving multi-objective optimization problems, named EGC-CMOPSO. The enhanced grid mechanism involved in EGC-CMOPSO is designed to locate superior Pareto optimal solutions. Subsequently, a hierarchical-based clustering is established on the grid for improving the accuracy rate of the grid selection. Due to the adaptive division of clustering centers, EGC-CMOPSO is applicable for solving MOPs with various Pareto front (PF) shapes. Particularly, the inferior solutions are discarded and the leading particles are identified by the comprehensive ranking of particles in each cluster. Finally, the selected leading particles compete against each other, and the winner guides the update of the current particle. The proposed EGC-CMOPSO and the eight latest multi-objective optimization algorithms are performed on 21 test problems. The experimental results validate that the proposed EGC-CMOPSO is capable of handling multi-objective optimization problems (MOPs) and obtaining superior performance on both convergence and diversity.

The rest of this paper is organized as follows. Certain algorithms are introduced in Section "Related algorithms". Section "Methods" describes our suggested EGC-CMOPSO in full. The experiment results of EGC-CMOPSO and the eight latest comparison algorithms on 21 test problems are discussed in Section "Results". Finally, Section "Conclusion" makes a summary of this paper and put forward a prospect for the future.

Related algorithms
The existing MOPSO algorithms. The Pareto-domination-based approach is used in most MOPSOs to select leading particles. By integrating an adaptive grid mechanism and an adaptive mutation operator, Agrawal et al. 7 proposed an interactive meta-heuristic multi-objective optimization (MOO) for roulette selection. Carvalho et al. 8 coupled multi-objective control technique CDAS with PSO, and the proposed MOPSO-CDAS contributed to the improvement of convergence and solution diversity. To boost the convergence rate even further, Moubayed et al. 9 integrated Pareto dominance and decomposition approaches into the solution mechanism. In D 2 MOPSO, a new technique for leading particle selection and external archive maintenance is proposed. The concept of the crowded distance between objective space and solution space is also introduced in it, which can fast converge without the genetic operator. A novel AgMOPSO 10 directed by external archives is presented by Zhu et al., allocating relevant particles to each decomposed sub-problem for optimization. Due to the curse of dimensionality and poor performance in MOPs, the classical optimization algorithm is being considered. Lin et al. 11 put forward a balanced approach to estimating fitness to solve this challenge and increase particle selection pressure. The novel speed update equation, simulated binary crossover (SBX) and evolutionary search based on polynomial variation (PM) are employed in NMPSO to considerably increase the algorithm's performance.
Decomposition-based MOPSO decomposes MOPs into several sub-problems. Inspired by MOEA/D, Cain et al. proposed MPSO/D 12 , and the crowding distance is used in it to determine the fitness value to increase the convergence. A revolutionary multi-objective and multi-population co-evolution technique is proposed by Zhan et al.,named CMPSO 13 . To improve the convergence rate, the elite information in the shared archive is utilized to adjust the particle speed. Meanwhile, the elite learning strategy is adopted to update the external archive to promote variety and avoid local optima. To meet multiple conflicting objectives, Yao et al. 14 presented a co-evolutionary multi-group multi-objective optimization algorithm ECMSMOO. Furthermore, an endocrine incentive mechanism is embedded in it to avoid falling into local optimal. MMO-CLRPSO 15 proposed by Zhang et al. uses a clustering strategy based on the Euclidean distance of the decision space for population division. Jiang et al. 16 combined a convolutional neural network (CNN) with the decomposition strategy, and the proposed MOPSO/D-NET reconstructs NAS into a multi-objective evolutionary optimization problem.
In addition, the researchers increased selection pressure by introducing evaluation indicators. Common evaluation indexes include super-volume HV 17  The grid mechanism. gbest is not the only option available to lead the evolution direction of the population in MOPs. The grid mechanism is first incorporated into MOPs by Knowles et al. 21 , thus ensuring the diversity of non-inferior solutions in external archiving. Suppose to select promising particles in the external archive, take Fig. 1a as an example and try to select particles in different grids, thus, p 2 and p 3 cannot be selected simulta- Enhanced grid ranking. Traditionally, the Pareto dominance relation is used to select elite particles, but in the late iteration, the selection pressure is insufficient, leading to the concentration of most particles in the first Pareto front surface. The introduction of the grid concept can improve the diversity and convergence of the population effectively by sorting particles in the grid coordinate system. Taking advantage of the grid mechanism, enhanced grid ordering is proposed in this paper, and the strategy is described in detail below. www.nature.com/scientificreports/ Definition 1 (Partition number of grid coordinates). The grid is utilized in EGC-CMOPSO to relocate the position of particles in the objective space, which is transformed from the decision space. The design of div is critical in the iterative process of the population. Taking into account the impact of N and M , and div is changed adaptively as follows: where N denotes the number of particles, M denotes the number of objectives. Apparently, with the change of N and M , the partition number of grid coordinates is also adjusted adaptively. Increasing or decreasing the selection pressure dynamically allows for improved convergence to the true PF, effectively improving the convergence and diversity of the final population.
GrEA is the source for grid-related definitions. The upper and lower boundaries of the grid on the i-th objective can be determined: where div represents the partition number of grid coordinates on every dimension, minf i (x) and maxf i (x) represent the minimum and maximum values of x on the i-th objective, respectively. Definition 2 (Grid coordinates). Based on the above definition of div , up and lp , the grid coordinate corresponding to the particle x in the objective space is determined by: where G i (x) denotes the grid coordinate of i-th ( i = 1, 2 . . . , M ) objective of the particle x , f i (x) is the actual value of the particle x on the i-th objective and ⌊.⌋ represents the floor function. For example, in Fig. 1a, the coordinate of the particle p 1 is (0.6,5.0) , after p 1 is mapped to grid space in Fig. 1b, G(p 1 ) = (1, 5) . As a result, the essence of grid-based dominance is similar to Pareto dominance, consequently, x grid-dominates y ( y ≻ grid x ) can be defined as: To make the selection of an elite particle set more reasonable, that is, particles close to the Preto boundary are selected under the premise of ensuring the diversity of particles. Inspired by this, we propose three evaluation indexes based on grid coordinates, namely enhanced grid dominance ( EGD ), grid dominance ranking ( GDR ), and grid difference ( GD ). Among them, EGD and GDR are used to evaluate the convergence of particles, while GD is used to evaluate the diversity of particles, and EGR combines the above three indexes effectively.

Definition 3 (Enhanced fitness values).
Based on the above concept of grid domination, the enhanced fitness value of each particle in the grid coordinate system is evaluated by constructing the accumulation function of the dominant particles. EFitness is an index of convergence and lays a certain foundation for the subsequent judgment of enhanced grid domination. Specifically, each particle is assigned a EFitness , representing the number of grid-dominate particles, denoted as follows: where |.| denotes the number of elements in a set that meet the given conditions. Definition 4 (Enhanced grid dominance). Based on EFitness , EGD can be calculated directly: According to the definition, EGD is also a convergence index. Instead of merely examining the domination connection between particles, EGD is determined jointly by the set of particles dominated by the grid and the number of particles it dominates. The smaller EGD , the better, i.e., EGD(x) = 0 implies that no other particle can overpower it. That is, x is a non-inferior solution. Similarly, a higher EGD(x) means existing many particles y grid dominates x , and y dominates many particles at the same time, resulting in a larger sorted value of x in our proposed algorithm. www.nature.com/scientificreports/ Definition 5 (Grid dominance ranking). GDR is used to evaluate the convergence of particles. The sum of grid-dominated coordinate differences on all objectives is calculated as follows: Definition 6 (Grid difference assignment). Different from the EGD and GDR indicators of convergence defined previously, the GD index focuses on the representation of particle diversity, and the sum of the absolute grid coordinate differences on each objective is defined as follows: The larger the GD value, the better the diversity and the more uniform particles are distributed.
Definition 7 (Enhanced grid ranking). We can generate an index EGR that incorporates diversity and convergence fully by combining the three measures mentioned above: From the above definition, it is clear that EGD , GDR and GD are used to calculate EGR . Based on this, Algorithm 2 is proposed to realize the calculation of disG (lines 3-5) and EGR (lines 6-9). Finally, take them as a return to guide the selection of L (line 5 in Algorithm 1).
Although the three indices of EGD , GDR and GD balance convergence and diversity well, considering the values of GDR and GD maybe the same, and the number of particles is an integer, this results in the same GDR . As shown in Fig. 1b, particles p 2 , p 3 and p 5 have the same EGR = 2.80 , and the quality of these particles cannot be discriminated adequately in the grid. To solve this problem, the concept of hierarchical clustering is introduced in the following.
Hierarchical clustering strategy. In the field of MOPs, CA-MOEA 27 demonstrates the feasibility and superiority of hierarchical clustering, especially for MOPs with irregular PF. Considering that the positions of non-dominated solutions constantly change at different iteration stages, the clustering centers are generated adaptively based on the positions of particles, and the populations are dynamically divided. Then, combined with the previously proposed EGD in each cluster to reorder particles, eliminate the poor particles after reordering, and select the leading particles required for the next step. The implementation process of this strategy is shown in Fig. 2. Algorithm 3 depicts the specific implementation. www.nature.com/scientificreports/ To make the objective values of each dimensional particle comparable, the values of the particle on each objective are normalized to [0,1] before clustering (line 2), taking into account the varying effects of different dimensions on the final results: Cluster II Cluster III

Cluster II
Cluster III Cluster I p p 8 8 p p 5 5 p p 6 6 p p 7 7 p p 2 2 p p 3 3 p p 4 4

Cluster II
Cluster III The number of clusters www.nature.com/scientificreports/ where z i,nor represents the corresponding objective value after z i is normalized, z i denotes the i-th objective value of the particle, z i,min and z i,max denote the minimum and maximum value on the i-th objective of the particle z i , respectively. Next, particles are assigned to the nearest cluster center ( C d ) based on the Euclidean distance between each cluster center. Iterate through each cluster center again, and calculate the number of particles assigned to the cluster (lines 4-7). The number of particles assigned to the cluster is counted (line 9) by traversing each cluster center. If no particle in the cluster, the sorting value of the particle closest to the distance from the cluster center is assigned to 0 (lines [15][16]. If there are particles in the cluster, sort them according to the Euclidean distance to the clustering center ( C i ) and EGR inside the cluster, to get rn dis and rn egr (lines [17][18][19][20]. Since the two sorting values initially correspond to different objects, to make them comparable, the two sorting values before comprehensive sorting are normalized (line 19). Finally, according to the combination ranking value based on grid and clustering, the top ten are selected as the leading particles ( L ), and N/2 promising particles ( P ′ ) are selected for the next competitive learning (lines [22][23][24][25]. For simplicity, Fig. 3 is taken as an example. The comprehensive ranking values can be derived by integrating the normalized ranking: R(p 5 ) = 7 , R(p 6 ) = 1 and R(p 7 ) = 8 . That is, p 6 ranks first in the comprehensive ranking of the eight particles in the figure. Therefore, in Fig. 3, p 2 and p 6 with the first and second R values are selected as L . The top four R values, p 1 , p 2 , p 3 and p 6 are retained for the next competitive learning, because of the inclusion of more information on convergence and diversity.
It is worth noting that the size of L affects the convergence rate of particles, to a lesser extent, the convergence of the final solution set to a certain extent. The size of L sensitivity analysis is tested on the DTLZ1-3 and DTLZ7 in CMOPSO. Thereby, the size of L is set to 10 in our proposed algorithm.
Competitive PSO. The basic idea of competitive PSO is composed of two parts: determining the winning particle and guiding the update by the winner. p win is determined by comparison with the current particles in the elite particle set ( P ′ ). It is worth noting that, different from the classical PSO, no traditional external archiving mechanism is adopted in EGC-CMOPSO. Each update operation of the current particle p i is directly determined by p win . As a result, no additional computational resources and space are required to calculate and store local and global optimal particles. Algorithm 4 provides the pseudo-code of the competitive PSO mechanism.  Figure 3. Illustration of clustering-based selection with grid ranking. www.nature.com/scientificreports/ The first part is to select the winner particle. Two particles are randomly selected as competitors from L (line 3), e.g., particles lp 1 and lp 2 in Fig. 4. A particle in the elite particle set p i ∈ P ′ is taken as the particle to be updated, e.g., the particle p 1 in Fig. 4. The Euclidean distances of lp 1 , lp 2 to p i and the origin of the grid are calculated, respectively (line 4). The leading particle with the smallest combined distance is selected as p win (lines 5-8) to guide the update of p 1 .
The second part is the update with the guidance of the winning particle. The winner directs the update of the position and velocity of the current particle. Because the competition-based particle learning mechanism considers just the winner, not pbest and gbest , the traditional formulation for updating position and speed is updated: where v i (t + 1) and x i (t + 1) represent the speed and position of the i-th particle at ( t + 1 ) iteration, namely, the speed and position after p i competition update, r 1 and r 2 are two randomly generated vectors with values between 0 and 1, x win denotes the position of p win . Environmental selection. The generated offspring and the primary population are taken as input, and the optimal solution set is selected by the mechanism of environmental selection. For simplicity, the environmental selection mechanism in SPEA2 is used in ER-MOPSO to select N promising particles for the next generation. SPEA2's truncation strategy sequentially eliminates individuals with minimum distance from their neighbors, which is an effective method to promote population convergence and diversity. The Wilcoxon rank sum test is adopted to compare the results of the algorithms intuitively. The best performance metric in all comparison algorithms on the corresponding column test problems is marked in bold. The symbols "+", "−" and "=" indicate the results obtained by the comparison algorithms are significantly better, worse and approximate to EGC-CMOPSO, respectively, at the significance level of 0.05. On the ZDT3 test problem, for example, the fourth row in Table 1 indicates that the IGD values of the different comparison algorithms are worse than our proposed algorithm.

Performance comparison. The mean and standard deviation of IGD values of EGC-CMOPSO and other
comparison algorithms are represented in Table 1. For the universality of the algorithm considerations, the results of 21 test problems with two or three objectives are listed. Each row provides a horizontal comparison of the results of the ERC-CMOPSO and comparison algorithms on a particular problem, where the best mean of all algorithms in 30 independent runs is highlighted against a grey background. The longitudinal comparison of IGD values of ERC-CMOPSO on different test problems is shown in each column.
EGC-CMOPSO outperforms the comparison algorithms in most cases, yielding 11 optimal outcomes on 21 test problems, particularly on the DTLZ test suite. Hierarchical clustering is used in both EGC-CMOPSO and CA-MOEA, but EGC-CMOPSO performs better in 18 test problems compared to CA-MOEA. The reason for the different results may lie in the introduction of the grid mechanism based on clustering, to select particles with better overall performance to guide competition, indicating the necessity of the grid mechanism. Similarly, EGC-CMOPSO is more competitive than GrEA in 17 test problems. This is attributed to the fact that GrEA only considers grid advantages and has certain limitations in some special cases. While clustering-based with grid ranking is adopted in the proposed EGC-CMOPSO, with better competitiveness. PREA is a newly proposed evolutionary algorithm, that performs worse than EGC-CMOPSO on 14 test problems. Compare to NSGA-II + ARSBX, MOEA/D-CMA, Two_Arch2, NMPSO, and CMOPSO, EGC-CMOPSO shows its competitiveness on most test problems. Similarly, the experimental results of HV, SP and Spread metrics are shown in Supplementary Tables S3-S5 online, respectively.
The distributions of the non-dominated solutions are depicted in Supplementary Figs. S1, S2, and Fig. 5. Supplementary Fig. S1 shows the Pareto fronts obtained by nine algorithms and the theoretical Pareto front after 50 iterations on the ZDT3 problem. Clearly, the solutions obtained by EGC-CMOPSO are most consistent with real PF, and the distribution is uniform on the real PF. While the convergence and distribution of the solutions obtained by NMPSO are the worst, this may be due to too few iterations. Similarly, it can be seen from Supplementary Fig. S2 that EGC-CMOPSO is superior to other comparison algorithms on the irregular three-objective DTLZ6. In contrast, the solution distributions of CA-MOEA, MOEA/D-CMA and PREA have poor convergence, maybe because the convergence speed of the evolutionary algorithm is not as fast as PSO in the early stage. Due to the competitive learning mechanism, CMOPSO performs poorly when compared to other PSO algorithms. This is possibly due to the fact that the winning particle guiding particle updating may not be the optimal particle, and hence cannot guide particles to quickly get closer to real PF at the beginning of the iteration. For the three-objective WFG6, EGC-CMOPSO is shown to be competitive in Fig. 5. While most of the solutions obtained by the algorithms converge to PF, the solutions obtained by EGC-CMOPSO are more uniformly distributed.
The PFs obtained from Fig. 5 show that the algorithms have largely converged after 50 iterations. Therefore, we compared the time required for all algorithms to iterate 50 times on test problems with different PFs. The experimental results in Table S6 of the supplementary material reveal that the clustering operation increases the computational burden of EGC-CMOPSO and the time required for iteration becomes larger, which is a drawback of our proposed algorithm.

Comparison on convergence.
To test the performance of EGC-CMOPSO on two-objective and threeobjective problems, the index change curves of EGC-CMOPSO and eight comparison algorithms on ZDT3 and DTLZ6 are shown in Supplementary Figs. S9 and S10 online, respectively. Our algorithm is illustrated by a curve with a red five-pointed star among them. As shown in the figures, IGD, SP and Spread of EGC-CMOPSO are the smallest among all comparison algorithms in most cases. Similarly, taking Fig. S10b as an example, with the increase in iteration times, the HV value of EGC-CMOPSO always remains optimal or suboptimal.
It is worth mentioning that, the convergence rate of EGC-CMOPSO is significantly faster than other algorithms prior to 50 generations. This indicates that EGC-CMOPSO inherits the advantages of PSO with fast convergence speed in the early stage, and based on it, integrates enhanced grid and clustering mechanisms to maintain particle diversity well.
Influence of the number of clusters. In EGC-CMOPSO, the number of clusters ( N c ) and grids ( div ) are introduced to implement clustering-based with enhanced grid ranking. To explore the sensitivity of EGC-CMOPSO performance to these two parameters, two related experiments are designed respectively. www.nature.com/scientificreports/ The performance metrics of EGC-CMOPSO with different numbers of clusters are shown in Supplementary Figs. S11 and S12 online. In the experiment, the population size is fixed at 100, the calculated resource is fixed at 1000 × N . N c is set in ascending order to 2, 10, 20, 50 and 100. To test the performance of EGC-CMOPSO on various challenges, the experiments are conducted on 4 two-objective and 7 three-objective test problems, respectively. In summary, when N c = 10, each performance indicator on different test problems is better. Too large N c results in too few particles in the cluster and affect the sorting selection. While too small N c results in too many particles in the cluster, which cannot make good use of the advantages of clustering. Therefore, N c is set to 10 in EGC-CMOPSO is the best option.
Influence of the number of grids. In studies to investigate the sensitivity of the number of grids, the size of the population is kept constant at 100, the computational resource is fixed as 1000 × N , and the number of iterations is 50. div is set to 5, 10, 20, 30, 50 and 100, the adaptive value of div obtained by using Eq. (4). The calculation of div used in EGC-CMOPSO is represented by a curve with a red five-pointed star. As evidenced by the results in Fig. 6, in most cases, the div obtained by Eq. (4) makes EGC-CMOPSO has better competitiveness on ZDT1, ZDT2, ZDT4, DTLZ6, DTLZ7 and WFG7. Therefore, it is feasible to use our custom div in EGC-CMOPSO.

Conclusion
This paper proposes a clustering-based competitive particle swarm optimization with the enhanced grid ranking algorithm. The grid of EGC-CMOPSO is constructed in objective space, and bottom-up hierarchical clustering is utilized to generate clustering centers adaptively. Each particle is assigned to the corresponding cluster. To assist in locating the leading particles inside each cluster, the normalized enhanced grid ranking is fused to conduct a comprehensive sorting for particles. The current particle is updated by the competitive particle swarm optimization based on the winner selected from the leading particles each time. Different from the other PSOs, the external archiving mechanism is not adopted in EGC-CMOPSO. Compared with the eight latest algorithms, EGC-CMOPSO is competitive on test problems with regular or irregular PF, and the obtained solutions strike a good balance of convergence and diversity. www.nature.com/scientificreports/ Given the increasing complexity of MOPs, our future research direction is to apply the clustering-based grid ranking mechanism to high-dimensional, large-scale, complex optimization problems with constraints. Furthermore, complicated MOPs in the real world deserve to be investigated, since EGC-CMOPSO has demonstrated its competitiveness in addressing MOPs.